IN THE CLAIMS : 

Please amend claims 1, 3, 5, 20 and 24, and 28 as follows. 

1 . (Currently Amended) An image processing device comprising: 

an image pick-up device having a fixed positional relationship with respect to a 

measurement object; 

an orientation sensor for measuring adapted to measure the orientation at an image 
pick-up visual point of said image pick-up device; 

a storage unit adapted to store calculation information to calculate the orientation 
and/o r of said measurement object or the position of said measurement object on the basis 
of an output from said orientation sensor; 

a prediction position calculation unit adapted to calculate a prediction position of an 
index in an image picked-up by said image pick-up device on the basis of the measured 
orientation; 

a target image setting unit adapted to obtain a prediction position of an index in an 
image picked-up by said image pick-up device using a m e asured value from said 
ori e ntation senso r and said calculation information stored in said sto r age unit, gene r ate an 
image with a peripheral ar e a around said prediction position in said pick e d-up imag e 
subjected to a r otational proc e ss on the basis of a r otational angle in a roll direction of said 
image pick-up device derived from said m e asured value, and output said gene r ated imag e 
as a target image extract an image area of the index from the picked-up image on the basis 
of the prediction position of the index, rotate the extracted image area using a roll angle, 
according to the measured orientation, of said image pick-up device, and output the rotated 
image area as a target image : 
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a detecting unit adapted to detect the position of said index in said target image by 
performing a template matching process between a template image of said index and said 
target image; 

an updating unit adapted to update said calculation information stored in said 
storage unit on the basis of a detected position of said index detected by said detecting unit; 
and 

a calculation unit adapted to calculate the orientation and/or position of said 
measurement object on the basis of said a measured value from said orientation sensor and 
said calculation information updated by said updating unit. 

2. (Cancelled) 

3. (Currently Amended): The image processing device according to claim 1, 
wherein said calculation information is the correction information to correct for an error in 
the measured value of attitude the orientation measured by said attitude orientation sensor, 
and said calculation unit calculates the attitude orientation of said measurement object on 
the basis of said measured value and said correction information. 

4. (Cancelled) 

5. (Currently Amended) The image processing device according to claim 1, 
wherein said calculation information is the correction information to correct for an error in 
the measured value of attitud e the orientation measured by said attitude orientation sensor 
and the position information of the image pick-up visual point of said image pickup device, 
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and said calculation unit calculates the position and attitude orientation of said 
measurement object on the basis of said measured value, said correction information and 
said position information. 

6. (Cancelled) 

7. (Previously Presented) The image processing device according to claim 5, 
wherein said updating unit updates the position information in the two directions except for 
a depth direction in the camera coordinate system of said image pickup device, even when 
an index of only a single point is detected in said detecting unit. 

8. (Previously Presented) The image processing device according to claim 3, 
wherein said correction information is the information to correct for an error in the azimuth 
direction among the measured values of the attitude measured by said attitude sensor. 

9. (Previously Presented) The image processing device according to claim 1, 
wherein said updating unit updates said calculation information on the basis of the detected 
position of said index in said picked-up image. 

10. (Previously Presented) The image processing device according to claim 3, 
wherein said updating unit updates said calculation information on the basis of a typical 
value of the updated value of said calculation information obtained for each index when a 
plurality of indices are detected in said detecting unit. 



1 1 . (Previously Presented) The image processing device according to claim 3, 
wherein said updating unit updates said calculation information on the basis of a 
dislocation between the prediction position and said detected position of said index in said 
target image. 

12. (Previously Presented) The image processing device according to claim 11, 
wherein said updating unit updates said calculation information on the basis of a typical 
value of said dislocation obtained for each index when a plurality of indices are detected in 
said detecting unit. 

13. (Cancelled) 

14. (Cancelled) 

15. (Cancelled) 

16. (Cancelled) 

17. (Previously Presented) The image processing device according to claim 1, 
wherein said measurement object is an image pick-up visual point of said image pick-up 
device and said image processing device further comprises a display unit adapted to display 
said pick-up image with the image in the virtual space superposed thereon on the basis of 
the orientation of position and orientation of said image pick-up device calculated by said 
calculation unit. 
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18. (Cancelled) 



19. (Previously Presented) The image processing device according to claim 1, 
wherein said measurement object is a visual point of the observer, and said image 
processing device further comprises a display unit adapted to display the image in the 
virtual space drawn on the basis of the orientation or position and orientation of the 
observer calculated by said calculation unit on a display screen, while optically transmitting 
the image in the real space through said display screen observed by the observer. 

20. (Currently Amended) An image processing device in which the position of an 
index in a picked-up image picked up by an image pick-up device is detected by template 
matching employing a template image of said index, comprising: 

an orientation sensor for measuring the orientation at an image pick-up visual point 
of said image pick-up device; 

a target imag e creating unit adapt e d to c re at e a target image having a pe r iphe r al 
area around a prediction position in said picked-up image subjected to a r otational p r ocess 
on the basis of the r otational angle in a roll direction from said measu r ed value by 
obtaining said prediction position of the ind e x in said pick e d-up image, using the measured 
value of said orientation measured by said o r ientation s e nsor, and outputting the targ e t 
imag e ; and 

a prediction position calculation unit adapted to calculate a prediction position of 
the index in an image picked-up by the image pick-up device on the basis of the measured 
orientation; 



a target image creating unit adapted to extract an image area of the index from the 
picked-up image on the basis of the prediction position of the index, rotate the extracted 
image area using a roll angle, according to the measured orientation, of said image pick-up 
device, and output the rotated image area as a target image: and 

a detecting unit adapted to detect the position of said index in said picked-up image 
by performing a template matching process between said template image and said target 
image. 

21. (Cancelled) 

22. (Cancelled) 

23. (Cancelled) 

24. (Currently Amended) An image processing method comprising: 

an image pick-up step, of picking up an image with an image pick-up device having 
a fixed positional relationship with respect to a measurement object; 

an orientation measuring step, of measuring the orientation at an image pick-up 
visual point of the image pick-up device; 

a storage step, of storing calculation information to calculate the orientation and/or 
of the measurement object or the position of the measurement object on the basis of the 
measured value measured in said orientation measuring step; 
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a prediction position calculation step of calculating a prediction position of an 
index in an image picked-up by said image pick-up step on the basis of the measured 
orientation measured in said orientation measuring step; 

a target image setting step, of setting a target image that is an object for detecting a 
predetermined index on the basis of the picked-up image of extracting an image area of the 
index from the picked-up image on the basis of the prediction position of the index, 
rotating the extracted image area using a roll angle, according to the measured orientation, 
of the image pick-up device, and outputting the rotated image area as a target image : 

a detecting step, of detecting the position of the index in the target image by 
performing a template matching process between a template image of the index and the 
target image; 

an updating step, of updating the calculation information stored in said storage step, 
on the basis of a detected position of the index detected in said detecting step; and 

a calculating step, of calculating the orientation and/or position of the measurement 
object on the basis of the a measured value from said orientation measuring step and the 
calculation information updated in said updating step. 

25. (Cancelled) 

26. (Previously Presented) A program code for executing the image processing 
method according to claim 24. 

27. (Previously Presented) A storage medium storing the program code according 
to claim 26. 
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28. (Currently Amended) An image processing method for use with an image 
processing device in which the position of an index in a picked-up image picked up by an 
image pick-up device is detected by template matching employing a template image of said 
index, said method comprising: 

an orientation measuring step of measuring the orientation at an image pick-up 
visual point of the image pick-up device; 

a target image creating step of c r eating a ta r get image having a peripheral area 
around a prediction position in the picked-up image subjected to a rotational process on the 
basis of th e r otational angle in a r oll di r ection from the measured value by obtaining th e 
prediction position of the ind e x in the picked-up imag e using the measur e d valu e of 
orientation measured in said orientation measuring step, and outputting the target image; 
and 

a prediction position calculating step of calculating a prediction position of the 
index in the image picked-up by the image pick-up device on the basis of the measured 
orientation measured in said orientation measuring step: 

a target image creating step of extracting an image area of the index from the 
picked-up image on the basis of the prediction position of the index, rotating the extracted 
image area using a roll angle, according to the measured orientation, of the image pick-up 
device, and outputting the rotated image area as a target image; and 

a detecting step of detecting the position of said the index in said the picked-up 
image by performing a template matching process between said the template image and 
said the target image. 
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